Data-driven effective model shows a liquid-like deep learning

The geometric structure of an optimization landscape is argued to be fundamentally important support the success deep neural network learning. A direct computation beyond two layers hard. Therefore, capture global view landscape, interpretable model network-parameter (or weight) space must established. However, lacking so far. Furthermore, it remains unknown what looks like for networks binary synapses, which plays a key role in robust and energy efficient neuromorphic computation. Here, we propose statistical mechanics framework by directly building least structured high-dimensional weight space, considering realistic data, stochastic gradient descent training, computational depth networks. We also consider whether number parameters outnumbers supplied training namely, over- or under-parametrization. Our reveals that spaces under-parametrization over-parameterization cases belong same class, sense these are well-connected without any hierarchical clustering structure. In contrast, shallow-network has broken characterized discontinuous phase transition, thereby clarifying benefit learning from angle high dimensional geometry. effective inside network, there exists liquid-like central part architecture weights this behave as randomly possible, providing algorithmic implications. data-driven thus provides insight about why unreasonably terms how different shallow ones.